1. Field of the Invention
The present invention relates to an improvement of a visual axis detecting apparatus, incorporated into an optical instrument such as a camera, etc., for detecting an axis of direction toward a gazing point at which a viewer watches, i.e., detecting a so-called visual axis (line of sight).
2. Description of the Related Art
There have hitherto been proposed a variety of apparatuses (e.g., eye cameras) for detecting which position on a viewing plane a viewer watches, i.e., detecting a so-called visual axis (line of sight).
In, e.g., Japanese Patent Laid-Open Application No. 1-274736, a parallel light beam from a light source is projected on an anterior eye part of a viewer's eye; and a visual axis is obtained by detecting positions of a pupil image and a cornea reflection image made by the light reflected from the cornea.
FIG. 1 and FIGS. 2A and 2B are diagrams showing a principle of the visual axis detecting method.
To start with, an explanation will be given with reference to FIG. 1. Respective infrared light emitting diodes (hereinafter abbreviated as IREDs) 13a, 13b are arranged in substantial symmetry with respect to an optical axis I of a light receiving lens 12. The IREDs divergently illuminate an eye of an eyeball of each photographer with light.
A cornea 16 of the eyeball 15 is illuminated with an infrared light beam from the IREDs 13a, 13b. At this time, cornea reflection images d, e with respect to a portion of each infrared light bean reflected by the surface of the cornea 16 are condensed by the light receiving lens 12. The images are re-formed in positions d', e' on the image sensor 14. Further, an image of the pupil of the eyeball 15 illuminated by the IREDs 13a, 13b is also formed on the image sensor 14.
Let xc be an x-coordinate of a center c of a circle (termed a pupil circle) having boundaries (a, b) between a pupil 19 and an iris 17, the x-coordinate on the image sensor 14 being determined as xc' (unillustrated).
FIG. 2A illustrates an eyeball image projected on the image sensor 14 of FIG. 1. FIG. 2B shows an output waveform of a video (image) signal transmitted from the image sensor 14 on a line (I)-(I') in FIG. 2A.
Referring to FIG. 2A, the numeral 50 designates a reflection image of a white part of the eyeball 15, and 51 represents a pupil reflection image. Further, the symbols 52a, 52b indicate cornea reflection images of a pair of IREDs 13a, 13b. These cornea reflection images are called Purkinje images.
Turning to FIG. 2B, two maximum points in a video signal 60 correspond to a pair of Purkinje images.
Referring back to FIG. 1, the x-coordinate of the mid-point between the Purkinje images d and e coincides with an x-coordinate xo of a center-of-curvature O of the cornea 16. Therefore, a rotational angle .theta. of the optical axis II of the eyeball 15 substantially satisfies the following relational expression: EQU (A1*L.sub.oc)*sin .theta..congruent.xc-(xd+xe)/2 (1)
where xd, xe are the x-coordinates of positions where the Purkinje images d, e are formed, L.sub.oc is the standard distance from the center-of-curvature O of the cornea 16 to the center c of the pupil 19, and A1 is the coefficient in consideration of a difference between individuals with respect to the distance L.sub.oc. Hence, the rotational angle .theta. of the optical axis II of the eyeball 15 can be obtained by detecting the positions of the respective characteristic points (the Purkinje images d, e and the center-of-pupil c) projected on a part of the image sensor 14 in the visual axis calculation processor. At this time, the above formula (1) is rewritten such as: EQU .beta.(A1*L.sub.oc)*sin .theta..congruent.xc'-(xd'+xe')/2 (2)
where .beta. is the magnification determined by a position of the eyeball 15 with respect to the light receiving lens 12 and substantially obtained as a function of a distance of .vertline.xd'-xe'.vertline. between the Purkinje images. The rotational angle .theta. of the optical axis II of the eyeball 15 is also rewritten by: EQU .theta..congruent.ARCSIN{(xc'-xf')/.beta./(A1*L.sub.oc)} (3)
However, EQU xf'.congruent.(xd'+xe')/2.
By the way, the visual axis of the eyeball 15 of the photographer is not identical with the optical axis II thereof. Accordingly, when calculating the rotational angle .theta. of the optical axis II in the horizontal direction, a visual axis .theta.H of the photographer in the horizontal direction is obtained by performing an angular correction .delta. between the optical axis II and the visual axis. The visual axis .theta.H of the photographer in the horizontal direction is given by: EQU .theta.H=.theta..+-.(B1*.delta.) (4)
where B1 is the coefficient in consideration of the difference between individuals with respect to the correction angle .delta. between the optical axis II and the visual axis of the eyeball 15. In connection with the symbol .+-., if an angle of rightward rotation with respect to the photographer is herein assumed to be positive, the symbol + is selected when the photographer looks in the viewing apparatus with the left eye, but the symbol - is selected when seeing through it with the right eye.
Further, the same Figure shows an example where the eyeball of the photographer rotates within a z-x plane (e.g., a horizontal plane). The detection is, however, similarly possible even in such a case that the photographer's eyeball rotates within a z-y plane (e.g., a perpendicular plane). However, a component of the visual axis of the photographer in the vertical direction coincides with a component .theta.' of the optical axis II of the eyeball 15 in the vertical direction, and, hence, a visual axis .theta.V in the vertical direction is expressed such as: EQU .theta.V=.theta.'.
Moreover, in accordance with items of visual axis data .theta.H, .theta.V, the positions (xn, yn) on a focusing screen of the finder field the photographer sees are obtained as follows: ##EQU1## where m is the constant determined by a finder optical system of the camera.
Herein, obtaining values of the coefficients A1, B1 for correcting the difference between individuals with respect to the eyeball 15 of the photographer involves the following steps. The photographer is made to fix the eye on a target located in a predetermined position within the camera finder. A position of the target is obtained by making coincident with a position of the fixing point calculated in accordance with the above formula (5).
Normally, the calculations for obtaining the visual axis and the gazing point of the photographer are executed by software of a microcomputer of the visual axis operation processor on the basis of the respective formulae given above.
When obtaining the coefficients for correcting the difference in the visual axis between individuals, the on-focusing-screen position of the visual axis of the photographer looking in the camera finder is calculated by use of the above formula (5). Data on the visual axis information is utilized for adjusting the focusing of the phototaking lens or controlling an exposure or the like.
The visual axis is actually obtained in the following manner. An eyeball image on the image sensor 14 is processed by the microcomputer or the like. The above-stated Purkinje images and the pupil circle (image) are detected. Based on positional data thereof, the visual axis is calculated.
A specific method is disclosed in Japanese Patent Laid-Open Application No. 4-347131.
The same Application shows the way of obtaining the pupil circle as follows. An eyeball video signal is read from the image sensor, and, meanwhile, a luminance difference of a boundary between the pupil and the iris is sequentially extracted as a signal edge, and its coordinates are stored. Then, when finishing the reading process of the eyeball image, a circle is estimated based on, e.g., the least squares method by use of a plurality of pupil edge coordinates stored. This circle is regarded as a pupil circle.
FIGS. 3A, 3B and 3C are diagrams of assistance in explaining an outline of this processing.
FIG. 3A depicts the eyeball image, wherein the Purkinje images are omitted. A plurality of white dots arranged around a pupil circle 51 are the pupil edge. The reference numeral 70-1 represents one of these dots. Further, FIG. 3B illustrates only the pupil edge of FIG. 3A.
The numeral 75 designates a circle estimated based on the least squares method by use of these items of edge data. FIG. 3C also illustrates this estimated circle, wherein (xc, yc) are the center coordinates of the circle, and rc is the radius thereof.
According to the conventional pupil edge extracting method, it is a common practice that edge characteristics are luminance variations in the eyeball image with respect to the iris and the pupil, and coordinates of an edge start point or an edge end point or a mid-point between the two points serve as pupil edge coordinates.
There was also invented another extracting method, as disclosed in Japanese Patent Laid-Open Application No. 3-177827, of determining the edge coordinates by use of Newton's method from data of respective points of the edge portion.
In any method, however, it can be said that the extracting method is very weak against noise.